The proliferation of online stores has led to users expecting high-quality and user-friendly presentations of products and service available for sale online. In their attempt to maximize profits and meet customer expectations, many online merchants may test different characteristics of the web pages of their online stores by performing A/B tests using the respective web pages.
A/B testing may be used to identify changes to web pages that increase a particular desired outcome. An A/B test is an experiment that has two versions, A and B, which are identical except for one variation that may influence a user's choice of action. During an A/B test, different statistically significant groups of users may be presented with different variations of an experiment to measure the users' responses to the respective experiment variations. Experiment variations may include differences in elements of a user interface presented to different users, wording differences in the messages addressed to the different respective users (e.g., “offer ends soon” or “offer ends Sunday”), etc.
The responses of the users to the respective experiment variations may be measured and used to generate test results (also referred to as “experiment results”) for each of the variations of the experiment. Examples of measurable outcomes of A/B tests are a click-through rate, the number of users registering to receive message from the online merchant, the number of sales made, etc. The test results of the particular experiment variations may be compared to determine which experiment variation is better at accomplishing a particular goal of the experiment.